Google DeepMind's Latest AI Agent Can Play 3D Games

Google DeepMind has unveiled its latest AI agent, capable of mastering 3D games with unprecedented skill and adaptability. SIMA, an AI agent designed to compete in games with the proficiency of professional gamers and execute tasks based on natural-language commands.

SIMA, an acronym for Scalable, Instructable, Multiworld Agent, has undergone training across a diverse range of video games. Essentially, the agent comprehends natural language directives pertaining to 3D environments and employs image recognition to carry them out.

Google has stated that “This work isn't about achieving high game scores. Learning to play even one video game is a technical feat for an AI system, but learning to follow instructions in a variety of game settings could unlock more helpful AI agents for any environment”.

Additionally, the company stated that they strive for their future AI agents to manage tasks requiring advanced strategic planning and multiple sub-tasks, such as locating resources and constructing a camp. This objective is crucial for AI as a whole because, although Large Language Models have produced powerful systems proficient at acquiring knowledge and formulating plans, they currently lack the ability to perform actions on our behalf.

Google collaborated with eight game developers, including Hello Games, Tuxedo Labs, Coffee Stain, Embracer and others. SIMA has been integrated into games such as Teardown, No Man’s Sky, Goat Simulator 3, and Valheim for training and evaluation.

As DeepMind continues to push the boundaries of AI research, the creation of its latest AI agent represents a significant step forward in the quest to develop truly intelligent machines. With its ability to play 3D games and adapt to new challenges, the AI agent offers a glimpse into the future of AI and its potential to revolutionize the way we interact with technology.